JPEX: A Psychologically Plausible Joint Probability EXtractor

Extracting redundancies in the data is the main purpose of unsupervised learning and estimating the covariance using Hebbian learning is a widespread way to achieve this. However, Hebbian learning only leads to the extraction of between-unit covariance. Because most associative memories use distributed representations, it would be more useful to extract the covariance of states. Yet, this operation would still be insufficient to fully model more complex environments, which include higher-order relations. In the present paper, we propose a new architecture, JPEX, which extracts higherorder joint probabilities at the state level using the tensor product as a learning rule. This new learning rule is compared with simple Hebbian learning in an environment which includes second-order relations. Also, JPEX’s ability to learn non-linear relationships is illustrated by training the model on the XOR categorization problem.